<i9)e#3#fw (jp) 02) £t 13 *|# tfr 3k (a) (nmmmm^m»^ 

#Hf§2000-340750 
(P2000-340750A) 

(43)&BB B ¥/ftl2*E12fl 8 B (2000, 12. 8) 

(51)IntCl. 7 IW3lje# FI f^3-r(##) 

H 0 1 L 27/04 H 0 1 L 27/04 V 5 F 0 3 8 

21/822 29/90 D 

29/866 



(21)tBW#*t 


J&BWU- 149315 


(71)tBJRA 000002185 








(22)mKB 


^11^5^280(1999.5.28) 


Stfcfi^JII&lfcfijn 6T@ 7#35^ 






(72)*^# *H «tfe 






SOSC^a/IIK^ftill 6TB 7^35^ V— 












(74) A 100086298 












F$r— A(t*%) 5F038 AVD4 AV15 CA05 CA10 EZ01 






EZ14 EZ15 EZ20 



(54) gm*>*m *w*&m<D®&*&&£ifi*mft&m 



(57) [Sft] 

[0«I] «tmt<D/c&5C*>y*^> (W) 

[»9b¥S] IMHRl 0(c»«3ftW»l©3>*^ 
h *-JH 1 y 1 4 a jWW* 6n/c¥iMf 

1C0^>^^ 1 1 rttcw^^l 4 a^MT 




1 

2<Dn>£^ H*wl>ftGc^y ^^^M^ai^t'- 
^S^coXfMch, 

mriam2CDn>^^ F#-JH*gCc, ^OiLfc->';n>® 

1^2cfc^^<^^CiCCJ:ff3, ^2(D^>^^h^ 

T*5^S»fl*^<t, BMSBi|fe»JB(C^3tifc02On 

K^*-;l^i£f&2<D:3>£^ htf-jKDJE 
CCD^2(Drj>^^ F*-;H*3CC, imi/c^'Jn>I 

&*> 6 r -/ - Fii*«3nt wci 
[iS3S14 3 BtTiBlf ^ F**— H(fi2#>^x^r 

^< x 9ic£.->xmmztxtck<Dx$,*), futa 

[000 1 ] 



C 2 ) ^12 00 0-34075 0 

2 

[0 00 2 ] 

z-rtc&L s 1 f^^^s^ftsgonsstft^ 

y*^— !f *t>^Ccfc*3 LS I CDfflaa**|B|-r 

/ctf><D#^*-- F v T&fcV^ if F 

&^ —^fC^NPN F5>^*£<£)X5 * 

[ 0 0 0 3 ] * fe % lsi «®4*«M*s«r 

20 [0 0 0 4 3 EI3, 14^ C(DJ:^^(DNPNh 

ic £ > ^> ^ *«Jfi£Sfl3 L/3fc*3»*«a©ies 
xeJBfcBiM-r-5fe»(OH"C** 0 JWT, C*i 

[0 0 0 5 3 *.f,.B3© (l ) tCvk-tXSlcPM&is 
*^ + ^JB3*JBeM-S 0 We. 03 co (2) Cc^Ti 

m^x. mz.te*vm (B) ^^^At^ci 

[0 0 0 6 3 'Xl>X\ m3<D (3 ) Cc^f^^Cc^y a 

-If t - FMffl««cr Kn>£^h7a 

BF, W«arYt>ttATS 
Ci&CcfcD. N P N h 5- ^MM^fC-^-X 8 ^ 

40 J&lS&ZLL&lzy^-Vv^m*- F»fiS««(c 

[0 0 0 71 ^-C, 133 (D (4) CC^-Tcfc^CC^R^ 
V> (P) £7U*t^ (As) ^-Y^>iiA"r^CitC 
<fct), NPN h^^iSXZBf&mmc^S 

1fv^-/*--K»a8«(|K:*V-F8a*»ttO, C 
tiicj:-oTNPN F^>^X^i^^^--tf 

- Fi^-en-enf#^ 0 ->y ^^skixcc^ 

^ KifiWR 1 0 

50 [0 0 0 8 3 H34 (£> ( 5 ) CC^-ril^CC^DCD 



(3) 

3 

If v^fVA*- Y<DT F8 a/*V- F9 alt^ti 
^nili;^02<Dzi>^^ F*-JH 2 £&y *-)l> F 

h*-;w l , ^2©3>5?i? h*-;n 2Ccoi>-c 

[0 0 0 9 ] X;*yt?&lcJ:-iXT i IS^^S: io 

jss&-r&c<t i 3£#*j&u cv 

1 4^r0^*T^ o 

C 0 0 1 0 3 yci>T\ f«S{Ufc^>^X^>Hl 4, 3? 
1 (6) CC^-T^^CC 

13*tt*Lt*l©3>^^F*-;H.l, 

h*— JH 2F^CC-en-enm#Jll 3 a^U 20 
^>^T~>^*^^ 1 4 a^Mt5 0 
[0 0 1 1)^0^ X^.y^ia^tTi^ 

JWcjsHKU -e<D?£. <2*aoy * x» 

0, StJCDT^^>^X^>^y 1 4 aic^T-ExJ:^ 
KILT. !34<£) (7) &C^T£0fc^';T>£^£tt& 
T iS^Al 5 <bA 1 £>Sm*A 1 jfrft&l 6<t£>e>& 

9, 7/-FI12 0, Flffi2 1 ^r^n^enjg 30 

[0012] c<D<t^c<:ort#en/c4^^^cc*o 

— Fm@2 0 <b*7V- F1S2 1 iOffltC^WTX 
WSlfctt, ^U7XW(a- 5 t7y-Fl®2 04' 

1CDS i ch#J£j£LTCfte<D^rt>6&&:7 4 40 

F#^3ft*Cd:fe:J:oTiBcS. 

[0013] 

X^W«, T^ — F^@2 0cDT<D£>^x«r>^ 
7^14a, 5 eSCi^<7)T<D3g#B 1 3 aJ^T^- F 
0^(7) A 1 <D# V- F9 affliJ^CD^ifr^P^'CO 
£5, CKDJ^CtLTA 1 <DfeW}1>m&tl2>±, Alt 

s i tfrhtzzv j 7^> F#^3ftct< <#9, @ 

3^Ll^^{C«T^- FH@8 ai^V- Ffg|E9a<L 50 



4$H 2000-340750 
4 

[0014] *»§8««fiB*t»«ca*rft:$*ifcfcO 

- f zmmmm u fcis ^ * ^ > h srsae u t»«r & 

[0015] 

&*^»#*^<!:. mtmmmtcmf&2titcm2<D^> 

mmizmi <d^>z? f*-;k H2<d:3>£>7 h^- 
^^ft^ft^u/cft, mzoz^isz? Y*->i>fo& 

>Ii^f}ciitsr^ s — ^ a& 6i^i7 ;u ^ ^ ^ 

p»wr/-F nm*Bf&-r z>m 3 cox^ii , 
[0016] co*z»tt«{a<o»«j&ffitc<fcn«, 

^r>Mf^l^. C<D^>^X^>r^2CDn> 
h^-;Vrt^:SS/cTCi^<^l <Dn>£^ 

ic v ^2<Dzj>^^ h^-;l/rt5c->'J n>S^K^®^^ 
U^^Cim, L/c3d^tC©S2^3>^^F 

^-^rttcr^- Fm^^-r^cifcjro, 

u /c ^> y cc^ii 3 & c i ^ ojtlic & & . 

[0 0 1 7 ] sfc. e>»;n> 

^ r-^A^^c^^r^^ -^A^^fe-Sr^- Fl 



5 

[0 0 18] C©*«fl«IfcJ:n«, 12©n>^^ 

FWfcWflu C*i5C<£D a 1 -S i &t>rj:z>y 

— ji/jWBjSSJvci^cd-c. c<D^2cDn>#>? h 
±fc $ l^Ci^O^u - ^ n*? & & CD 

[0019] 

[0 02 01 ^810(1) «c«rj:9«:ae*tiBi 
HfcteLT, PSOi/ y 3>ss l on p n h ^ 

« 1 -hOC Nix f£ * + ;Ujg 3 gflgiatr £ o 
[0 02 1 ] JXCC, 0 1 CD (2) fc^T^SCCLOCO 

sS(cJ:-3r7^-^PMfUI4««ttL. Mot, 01 
*tf*^jpt (B) i-f^^ttA-rscieci^rps* 

tcBf$<T&m2 zxis&2 b&—Jb& % 04cD ( 1 ) — 
(3) &C^L/tfi£^CDl&2CD:3>£^ h*-;H 2 debt 

5CLT, ?*-*FlMlJl4 

[0 02 2 ] 3CI»T. 01 CD (3) CC7j^J;5CCtE3fc£ 

EJfiMcbr, ^> y 3 >sk i tffi(c^ ^ 7 7 Re 
-b'^zzbi azmm-rz. acor. bf, £B5rg 

FJBjOMIRScr^- F8 a*Bf&-r2> 0 
[0 02 3] ^Ct»r % 0 1 (D ( 4 ) CC^-T J: 5 KfiE5fe£ 



W ^Pg 2000-340750 

6 

WWcUT, «A«y> (p) £fc&Sb3t (As) *A 

£«£fc&c, Vx^-V *^y-/*-FJBa««Cc*v- 
F9a^r^^L>, CtiiCJ: N PN i 

y ^ >g§^ i _b 5c ^ -f mmm i o 

[002 4]^^r, 02 CD (5 ) liCmT J: 5Sc££ijcd 

n^niiD^H l<7)rj>^>; h*-;H 1 — 
Y<DT F8 a/i77- F9aCt^*i 
-enjiD^02On>^^ h*-;U3 0 h&y h —>>P F 

<Da>** h*-^3 OCDifiW^, »(CfiSSCC»lSt 
£ # > ^>MCD51i^CD 2 ffi%M7LX]&f$ l '?Z e 
[0 0 2 5 ] * £££&Ccfc tTT i 

fi^T* elites 9 3S#/fi 3*jft&u «i>r, cv 

20 CDn>#>7 h*-jt,3 0|^£>y*^>^«fcTC£ 
£c<, H 1 CDn>^>> F^-;l/ 1 l^^>^f>"Cl 

[0 02 6] CCT\ CCD^>^X^>|il 4<DJS^t 
JCOUT^, IllfcJ; 5 Cti£Jit2 CD^J> ^ ^ h 
3 0CDifiW^^>^^^ >Ml 4©fflWt©2fi5*«il 

^2<D^>^^ h^h-;U3 OCDiflWCD iy2ct0«< & 

30 >iu <Dmm t <t mmcummm i 3 cdm;¥^ *> 

^7">«l 4*(t<DfltflECCj:k^r-H>fC»< v mW&) 
iCte2>>fXT>mi 4#flMDJBW*iCCD5r>^^> 
JR 1 4i|M*CDJK/f£i?g#jt 1 3 <DjByi<bCD^ftCCi2^ 

[002 7] C©J:^{CLr«fll3, *>^Xir> 

mt<D^m %m2<D^>2# h^-)i3 ocdibwcdi 

/2 J:f3»<^l>-Cl^Cch^6, 02<£> (5) {C^ 

L/cJ:^{c02cDn>^^ h^-;i/3 Oftrte, mf^L 
/c c fc ; 5fCCCD||2CDn>^^ F*-;|/3 Oft^^>^x 

^ > ?xr>&mm u torn*. 

^ 02 8 ]^r, f^L/c^>^X^>jgil 4, ® 
mm I 3«x?^t 9 ^LrS2<D (6) CC7jrrJ:5K 

^ -f -^FietMii o_bcD^>^XT->jiai 4, 

50 13^iSt^ B CCD^^CCb-Cx^^^'^ >/£ff ^ 



7 

£ s mi<D^>Z? F*-;H l^cSfll 3 eL&ft 
flKfca»»l 3i^>^7->il 4£;^£>ft&lW 

F^^aiwtsiiims, ft*>\ ccDct 

3 0F*9CCi^^ F^*-JU3 1 £ff*/£T&<t, C(DS2tD 
3>£* F*-;b3 OrtCDJSSIWJ, F^*-^ 

1,31 bbcc-c^u ^ i <DSiM#^fcti-r& 0 

*-;l/3 0Ftyc£>^X^>£»fc£ftl>J:5CCl,r £ 

>mi 4<Dmmticm$wis< fto. ofca^-c?*- 

^hmil 0±cd^>^x^>M1 4£6fc5£ir£J:0 
CCU-Ci. ^"T&C iiCcfcO, ^2<D33>^^h 

[0030] ycu-c\ fie*iigacci/r^^ » *ss*r 

^2<D (7) tCijVr«£5CON*y T^£Jt-<!:ft£T 
il5iAl £>£t>&*A 1 3fc£^l 6 <b#>6ft&"<-;* 

^>^«l^M#^ttiLTl,>£CD~t\ C©Hffil/c>»; 
n>S^mTOc^3lO. Sfccc?* wuFilftliWKl 0 
±{C*^SCXfc«3i8r, Ti^il5aiAU^ 
&*A 1 6 a tfrhtl&T Fmms 2 % te<£ 

zffjv- K«ffi3 3^^n^n?f^$n, ctuc^* 

[0 0 3 2 ] C©J;5ft¥aW^»c*oTtt, 

-if ?:/#^*-Fr*©^*wr&*»^ S££<d 
*><z> <t nfliccmHiffr t * * * !f ar ^ y -r f 

(D7^-K8a<b^V-K9a<!:^, -Tftt>^T^- 
F*S3 2 i*V- F*S3 3 &CDntC&'Wr;*ftEP 
»HU Cti6OP^^K«^*&5i±^ 0 t^i, ##J(D*> 
CDCC*Jti-C«, 7/-Kli3 2^^>^f >7*7^ 

^*r^c<hft<m2(D33>^^ h^-;u3 outers 

A/5^9A*q$ACC*7V~- F9 afllJCC&ttU cncCcfc 
0 A 1 - S i iPht£Z>V A b&&mi>xmf&2ft 



(5) ^P§2 000-340750 

8 

££fc<D£ft£ 0 

[0 03 3] ^2<DZJ>^^ h^-;b3 OtC^> 

^*^>3&>6ft£U--r F^*-Jl/3 1 £ff£j£L/tCDT\ 
C<Dg!2CD:3>£^ F*-;U3 0f*glCfl2j£OfcT F 
*B3 2co*^U-^*A»«:-r*ci3W*C». Lfc# 
oT^U-^AtceH-r^r F«K3 2 OIKS 

[0034] ft**, tne^MisttWra. *#mb*n p 

lcm.j£2ti2>C <hft< , npn F^>^*£te:f^TP 
MOS^NMOS, CMOS^CC>#^i*^^{lt^^ 

[0 03 5 ] 

20 Xf>ti2©n>^^ h*wi/rt£«fc-rc<tft<lfr 
l <£>^>£ v h*-;v€r£>^;*^>rS&i*tF<fc5K: 

\stctt3kX2bZfrh, ^Of££>y*:r>^x*^> 

ft< CCDT^- F««ft*aL/yfcVi;n>*fiSffl6c» 
30 [0 03 6] J:ot, c<Dcfc^ccurt#e>nfc*^(D 

FfflKc»»U C*UcJ;DA 1 -S ifrhte&y 4 U 
[0 03 7] ^^>CC, ^2©rJ>^^ F^-;KD{HiMCC 

40 ^>^XT*>^^^^tf^ F^*-;v*W^S^nrc^ 

&*)tfj*\s-*j> tmitf ft & o £ ft x mmv'Cimw ft c ^ 

^(DiftO, ^feif ^ F^*-^gft^>m®£urtR^ 

feRSih^n/c^CDift^, L/^-7tC(D*#»gg(J 
{fffit4CDliSC^CD<Lft^ 0 
[aBOfflf^ftlttW] 

[in < i ) - ( 4 ) *mA<Dmmwmw<Dm& 

so *WKlKW*xa««cttM^&fcfeo»«raK 



1 



9 

im 2 ] ( 5 ) - < 7 ) w\ ^qao^»«»flicr>»is 
mi ] 



(6) 2000-340750 

10 

* [B4] <5> - <7> I*. $t&<D¥m»mm<D9my} 
m<D—m*mm?zfcti><Dmx~$>*). m3co (4) sa& 

1-S/'J3>IR 1 11-* 
1 coa>^^ F^-iU, 14-^>^xf>E 14a 

[H2] 



CI) <M P N Tr> 



<2ener Diod«> 



(2) 



(3) 




(4> 



<N PN Tr> <Zen«r Oiode> 

117 6 \*~* 10 J 13 / 




6 1 7a 3 8a 9a 5 

31 U"-f h*9r-JI/ 30 4 




8 9 2 1 3 31 3 7e aa g a 5 



h-3 BLANK (uspto) 



Searching PAJ,, 



http://wwl9.ipdl.ncipi.go.jp/PAl/result/detail/main/wAAAD8. 



PATENT ABSTRACTS OF JAPAN 



(ll)Publication number : 2000-340750 
(43)Date of publication of application : 08.12.2000 



(51)Int.CI. 



H01L 27/04 
H01L 21/822 
H01L 29/866 



(21) Application number 

(22) Date of filing : 



11-149315 
28.05.1999 



(71) Applicant : 

(72) Inventor : 



SONY CORP 
OISHI TETSUYA 



(54) SEMICONDUCTOR DEVICE AND MANUFACTURE OF THE SAME 
(57)Abstract: 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **** s hows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the robot control system to which a robot is moved, 
starting the robot control system for carrying out the remote monitor of a plant, the works, etc. using a 
robot, especially uniting autonomous control and remote control. 
[0002] 

[Description of the Prior Art] When the mobile robot for carrying out the remote monitor of a plant, the 
works, etc. was used from the former, there were an approach of roughly dividing and moving all mobile 
robots by remote operation, and a method of making of operation [ all ] give a mobile robot 
autonomously. 

[0003] although the motion of a robot is fundamentally supervised with the TV camera etc. by the 
approach by remote operation, since there is little information acquired compared with viewing in a TV 
camera, compared with the case where it is accepting reality, it is markedly alike, and actuation is 
difficult and an operator's burden is also large [ actuation ]. 

[0004] Therefore, a mistake arises by the immaturity of an operator's technique, fatigue, etc., a mobile 
robot cannot avoid risk of an obstruction etc. or there is a possibility of colliding with surrounding 
people and a surrounding object. Moreover, if the situation where the communication link with a mobile 
robot is stopped in cutoff of the electric wave by failure of a communication device, a building, etc. also 
produces the skilled operator, stop instruction for risk aversion etc. cannot be given to a mobile robot, 
but even if sensing will be carried out a mobile robot conversely (a surrounding image, contact on a 
body on the street, etc.), un-arranging [ that an operator cannot recognize it ] will arise. 
[0005] By furthermore, the reasons nil why low-cost-izing or infrastructure development of the means of 
communications of an operator and a mobile robot are not ready »etc. If communication link delay arises 
by transmission through two or more paths etc. when means of communications with small channel 
capacity is being used for example Since an operator has to give the command of passing speed, a 
direction, etc. based on a mobile robot's sensor information (a robot location, bearing, image 
information, etc.) transmitted behind time, the command will also be overdue and a mobile robot will be 
reached, The operability of the real time to a mobile robot gets very bad. Therefore, when sudden 
obstructions, such as a pedestrian, appear on a mobile robot's path, a mobile robot cannot be stopped 
immediately. 

[0006] On the other hand, when making a mobile robot do autonomous working, the dependability of 
migration has required the mobile robot itself for the ability to recognize [ which ] a surrounding 
environment correctly, and the following points pose a problem. 

(1) the dependability of sensing — for example "Stentz, A., Hebert, M., and "A Complete Navigation 
System for GoalAcquisition in Unknown Environment" and Proc.of IEEE International Conference on 
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Intelligent Robots and Systems'95", Although the example of an experiment which carries out 
autonomous migration is stated to pp.425-432 and 1995" (reference 1) while a vehicle mold mobile 
robot uses a laser distance sensor for the inside of a strange environment and detects a surrounding 
failure When distance detection is not completed on the water surface etc., it is necessary to operate it by 
human being wedging himself in autonomous control of a mobile robot. 

[0007] moreover, to the trend of the autonomous mobile robot research in "carat appearance Takeo and 
the U.S., JRSJ, Vol.5, No.5, pp.41 -51, and 1987" (reference 2) Although the example which carries out 
autonomous migration of the same vehicle mold mobile robot in the environments (the road where 
pavement is imperfect, the road which has a shadow on a road surface, road where fallen leaves exist) of 
an outdoor indeterminate is described The travel speed is not it being as late as 0.6km/about h, and 
moving by any scenes to stability moreover. 

(2) Although the route when carrying out autonomous migration of the vehicle mold mobile robot from 
a starting point to the destination is shown, the incompleteness 1, for example, the reference, of a 
migration plan, for trial-and-error, the detour was performed repeatedly and the useless motion of the 
motion of the first half has increased. That is, what can advance to the destination that human being 
taught the root more simply [ way ] may happen. And the algorithm for generally generating the root in 
autonomous migration is complicated. 

(3) Although connected also with an expensive sensing system (1), in order to raise the dependability of 
sensing from the former, to unite sensor information of a different kind (sensor fusion) is tried. 
However, generally the cost in connection with sensing increases, arid a system becomes expensive. For 
example, although the sensor system with a precision of 10mm which combined GPS (Global 
Positioning System), the engine-speed sensor (encoder) of a wheel, and the inertia sensor unit (three 
accelerometers, three gyroscopes) as a terrestrial location detection means is formed in the vehicle mold 
mobile robot shown in reference 1, the cost is about 1 millions (about 120 million yen). 

[0008] By the way, in order to cope with it inconvenient [ actuation of the mobile robot only by remote 
control which was mentioned above or autonomous control ], in recent years, try, both sides are made to 
specifically share knowledge, and the attempt whose operator and robot are made to cooperate mutually 
and with which autonomous control and remote control are united according to an activity is made. 
Making an operators burden light and attaining the increase in efficiency of an activity by this, is 
expected. 

[0009] For example, autonomous remote fusion control of a double arm hyperdactyly form manipulator 
besides "Shimizu (the 1st news), The Robotics Society of Japan academic lecture meeting, pp.263-264, 
1990" (reference 3), "Autonomous remote fusion control of a double arm hyperdactyly form 
manipulator besides Omichi (the 2nd news), Japan Society of Mechanical Engineers ROBOTTIKUSU 
mechatoronics lecture meeting'91 lecture collected works (Vol. A), "pp.49-50 and 1991" (reference 4), 
the activation-teaching method in autonomous remote fusion control besides Omichi, To the Robotics 
Society of Japan academic lecture meeting, pp. 569-570, and 1990" (reference 5) The example which 
applied such an autonomous remote fusion control robot to the double arm hyperdactyly mold master 
slave robot is described, and there is a publication about spatial fusion and time fusion as an approach of 
combining autonomous control and remote control. 

[0010] It is possible spatial fusion to use some degrees of freedom of a manipulator for remote control, 
and to be the approach of using the remainder for autonomous control, for example, to make the left arm 
of a manipulator into autonomous mode in slinging-work actuation, to make a hook grasp, to make a 
right arm into a remote mode, and to make a wire hung. 

[001 1] However, the base itself is moving and, moreover, unlike actuation of a manipulator, a mobile 
robot cannot catch a sight of the whole mobile robot from outside. Therefore, a role assignment of 
autonomous control and remote control cannot be easily performed like the manipulator mentioned 
above. Moreover, it is very difficult for a surrounding environmental variation to predict a surrounding 
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situation in a manipulator, since a surrounding environment changes every moment by the mobile robot 
to there being much self, i.e., the thing which arises as a result of the activity of a manipulator, and being 
able to predict. 

[0012] The above problem forms a camera in the interior of a mobile robot temporarily, and since it is a 
problem produced even if it operates by remote control, while an operator looks at the image sent by the 
mobile robot, it cannot form the fusion plan of autonomous working and remote actuation beforehand 
like a manipulator. Therefore, it can be said that it is difficult to perform spatial fusion in a mobile robot. 

[0013] On the other hand, time fusion is the approach of uniting remote actuation for autonomous 
working in time according to a robot's working hours. It is also possible for there to be a concept of a 
time dial in this time fusion, as shown in reference 2, for example, to turn back actuation of a 
manipulator in time. However, since the base is being fixed, a manipulator can return the same hand 
orbit, but since there is no base fixed in the mobile robot, there is a problem that the same orbit cannot 
be returned. 
[0014] 

[Problem(s) to be Solved by the Invention] Although there were two moving methods of migration only 
by remote operation and migration only by autonomous working in the conventional mobile robot as 
mentioned above, there was a problem that actuation is difficult for the former, and it was difficult to 
cope with the situation where generating of an obstruction etc. is insufficient, appropriately since it may 
be able to stop being able to operate it on real time according to a communication link condition. 
Moreover, the latter had a possibility that a robot could stop being able to operate appropriately and a 
useless motion might increase, and had the problem that a sensing system became expensive. 
Furthermore, applying the fusion approach of of autonomous control and remote control, such as spatial 
fusion and time fusion, to a mobile robot had the problem of being difficult. 

[0015] This invention is for solving such a problem, autonomous control and remote control can be 
combined appropriately, it is cheap and an operator's burden aims at offering the robot control system 
which can also cope with the insufficient situation easily few moreover. 
[0016] 

[Means for Solving the Problem] In order to solve the above-mentioned technical problem, this 
invention is a robot control system to which this robot is moved by performing data communication 
between a robot and an operating set. An operating set A display means to display the data transmitted 
by the data and the robot which were set up beforehand in order to teach a robot, It has an instruction 
means for teaching the moving-target data of the action module unit which should be transmitted to a 
robot on the display screen of this display means. A robot A knowledge base storage means by which 
the knowledge base for the autonomous migration set up beforehand was memorized, A decision means 
to judge whether the autonomous migration based on the knowledge base memorized by this knowledge 
base storage means is possible, When it is judged by this decision means that the autonomous migration 
based on the knowledge base is possible, autonomous migration is performed based on the knowledge 
base. When it is judged that the autonomous migration based on the knowledge base is impossible, after 
degenerating current autonomous migration, It is characterized by having an autonomous migration 
means to perform autonomous migration based on the moving-target data to which the situation data in 
which a current situation is shown were taught by the operating set with the instruction means based on 
delivery and this situation data. Here, in order to teach a robot at a display means, the data transmitted 
by the data and the robot which were set up beforehand are displayed on two or more coincidence, 
respectively. 
[0017] 

[Embodiment of the Invention] Hereafter, the operation gestalt which applied this invention to the 
autonomous remote fusion control system with reference to the drawing is explained. 
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(1st operation gestalt) Drawin g 1 is drawing having shown the appearance of the autonomous remote 
fusion control system concerning the 1st operation gestalt, (a) expresses the robot which moves the 
inside of a plant etc., and (b) expresses a robot's operating set. Moreover, drawing 2 is the block diagram 
having shown a robot's configuration, and drawing 3 is the block diagram having shown the 
configuration of an operating set. 

[0018] This autonomous remote fusion control system consists of communication devices 3a and 3b for 
roughly dividing and communicating between a robot 1, an operating set 8, and a robot 1 and an 
operating set 8. A robot 1 operates by autonomous remote fusion control with which autonomous 
control and remote control were united. Namely, a subject of operation and operations sequence are set 
as the robot 1 as the knowledge base (environmental model). When a robot 1 operates autonomously 
while the part in which autonomous working is possible took the check to the operator based on the 
environmental model, and lack is in the information on an environmental model, That is, when it cannot 
respond only for the information on an environmental model, such as generating of an obstruction 14, 
and lapses into the condition in which its action decision is impossible, that is communicated to an 
operating set 8 and an operator is asked for subsequent concrete directions. If an operator inputs the 
directions according to this with an operating set 8, the instruction data is sent to a robot 1, and a robot 1 
will make the instruction data reflect in an environmental model, and will resume autonomous working. 
In addition, same processing is performed also when performing the instruction of a robot 1 between 
which an operator intervenes free working (activation-instruction), for example, the instruction 
equivalent to correction of a location posture. 

[0019] Hereafter, the configuration of each part is explained to a detail. As shown in drawing 1 (a), the 
robot 1 is constituted as a wheel mold robot which moves by four wheels 5, and CCD camera 2, the 
distance robot 4, the encoder 6, and the gyroscope 7 are formed as a sensing means. That is, a robot 1 
incorporates surrounding image information with CCD camera 2 as sensor information, detects the 
distance to the obstruction 14 which exists ahead in predetermined distance by the distance robot 4, 
detects the rotation of a wheel 5 with an encoder 6, and detects the direction of a body (posture) with a 
gyroscope 7. 

[0020] Here, it is easy to be the thing of extent which can recognize the obstruction 14 which exists on a 
robot's 1 course at least as a distance robot 4, and it cannot be necessary to recognize clearly what the 
obstruction 14 is. For example 'Tllah R.Nourbakhsh and David Andre, CarloTomasi, and and To 
Michael R.Genesereth, "Obstacle Avoidance Via Depth From Focus", and ARPA Image Understanding 
Workshop 1996" (reference 6) Although the method of planning distance from the dotage condition of 
an image as a cognitive system is indicated By photoing the image of a front fixed distance by the 
distance sensor 4 similarly, and detecting the distance from the dotage condition of the image to the 
obstruction in an image, in the recognition precision of an obstruction 14, although it is inferior, positive 
sensing is realizable. However, when using an ultrasonic sensor, since detection when the reflector of an 
obstruction 14 inclines is difficult for a supersonic wave, it is desirable [ when using a laser range finder 
as a distance robot 4, detection of the distance over the obstruction 14 of a black field is difficult for 
laser light, and / a supersonic wave ] to use other sensors together. 

[0021] Communication device 3 a for communicating with an operating set 8 if needed is prepared for 
the robot 1 . In addition, since an operating set 8 only gives the command of the action module unit 
(action level) of the robots 1 which do not give the command of movement control level on real time to 
a robot 1, for example, show the destination on a 2-dimensional map, such as coordinate data, as 
communication device 3 a, real time nature, such as PHS (Personal Handyphone System), may not 
necessarily be guaranteed. 

[0022] Furthermore, the picture compression section 21 for compressing into a robot 1 the image 
photoed with CCD camera 2 to be shown in drawing 2 , The knowledge base section 23 the knowledge 
base for the autonomous migration set up beforehand (environmental model) is remembered to be, The 
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decision section 24 which judges whether the autonomous migration based on the knowledge base is 
possible, and the dead REKONINGU section 25 for measuring a robot's 1 center-of-gravity location 
with autonomous migration and remote migration are formed. In addition, in this drawing, the control 
device 22 is formed by the knowledge base section 23, the decision section 24 and the dead 
REKONINGU section 25, and communication device 3a, and it considers as migration of a robot 1 and 
the thing by which the rotational frequency of each wheel 5 is specifically controlled with this control 
device 22. 

[0023] On the other hand, as shown in drawing 1 (b), the operating set 8 is constituted by 
communication device 3b, the image display machine 9, data storage 10, the data-conversion processor 
1 1, the mouse 12, and the loudspeaker 13. 

[0024] The 2-dimensional map information (line drawing) corresponding to a mobile robot's 1 
successive ranges (for example, site of works etc.) set up beforehand be memorize as data required for 
data storage 10 in order to teach a robot 1 the migration root, and the 2-dimensional map corresponding 
to a required part be display by an operator's selection on the screen of the image display machine 9. 
[0025] The image display machine 9 is for displaying the static-image information sent by the robot 1, 
displaying the independent image of the 2-dimensional map information (line drawing) memorized by 
data storage 10, or displaying these compound images by multi-window. Here, the image display 
machine 9 shall make a display image zooming or the thing which is made to advance side by side and 
rotate and can be displayed free, and shall perform selection of the display image of business from from 
among two or more display images. Moreover, the image display machine 9 is used also in the case of 
the moving trucking instruction to a robot 1 . Namely, as shown in drawin g 3 , the pointer P on the 
screen of the image display machine 9 can be operated with a mouse 12, and directions of the 
destination etc. can be given to a robot 1 now. 

[0026] The data-conversion processor 1 1, performs zooming of line information, such as the focus on the 
2-dimensional map memorized by data storage 10, and a border of a road, coordinate transformation 
processing of advancing side by side and a rotation, etc. 

[0027] Hereafter, the processing when moving this robot 1 by autonomous remote fusion control is 
explained. First, a robot's 1 destination is directed by the operator in an operating set 8. Specifically with 
an operating set 8, a 2-dimensional map is independently displayed on the image display machine 9 
based on the 2-dimensional map information memorized by data storage 10. An operator inputs a robot's 
1 destination into an operating set 8 by clicking one on the 2-dimensional map displayed on the image 
display machine 9 with a mouse 12. Thus, the coordinate data on the 2-dimensional map showing the 
destination of the inputted robot 1 is sent to a robot 1 by communication device 3b as instruction data 
through the data-conversion processor 1 1 . However, a robot's 1 destination may be beforehand set as the 
knowledge base section 23 by the side of a robot 1 in sequence. 

[0028] A robot 1 makes the instruction data received by communication device 3a reflect in the 
knowledge base, based on the knowledge base, to the destination, unites autonomous migration and 
remote migration and moves. That is, a robot 1 moves by autonomous migration fundamentally based on 
the contents of a setting of the knowledge base section 23 (2-dimensional map information, the criteria 
information on migration, etc. are included besides the migration root). And only when action decision 
is itself impossible, an operator gives a robot 1 directions. 

[0029] Specifically, a robot 1 does autonomous migration by carrying out dead REKONINGU from a its 
present location to the destination at the shape of the shape of a straight line, and a curve (for example, 
radii) set up beforehand. That is, a robot's 1 dead REKONINGU section 25 measures a robot's 1 center- 
of-gravity location based on the data of the direction of a robot 1 (posture) detected with the rotation and 
gyroscope 7 of each wheel 5 which were detected by the encoder 6. A control device 22 controls the 
engine speed of each wheel 5 based on the measurement result of the dead REKONINGU section 25, 
and a robot 1 does autonomous migration as the result. 
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[0030] Moreover, in order to correct the error by slipping of a wheel 5 etc., it is made the criteria of a 
robot's 1 migration of suitable things, such as a curbstone path on the street, and may be made to move 
by learning to the criteria. The indicator formed in along a manhole besides the curbstone which was 
learned and was mentioned above as criteria of migration, and a road, the white line drawn on a path on 
the street can be considered. 

[0031] Thus, while performing autonomous migration, by the robot 1, it judges whether instruction of 
action is required of a being [ performing autonomous migration succeedingly based on the knowledge 
base / possible ], and operating set 8 side. Specifically, a robot's 1 decision section 24 degenerates the 
action at the time to the level which can once secure insurance as that which lapsed into the condition in 
which the action decision (autonomous migration) only by the robot 1, i.e., the knowledge base, is 
impossible, when it judges whether an obstruction 14 exists in the fixed distance on a course from the 
detection result of a distance robot 4 and the obstruction 14 has be recognize. In this case, it is not 
necessary to recognize what that obstruction 14 is concretely that what is necessary is to recognize that 
an obstruction 14 exists as mentioned above. 

[0032] A robot 1 uses a rate as degeneration of action. That is, a control device 22 secures insurance 
when an obstruction 14 exists on a course by controlling the rotational speed of each wheel 5 and 
adjusting the passing speed of the robot 1 whole. At this time, if a robot's 1 rate is made into zero and 
stopped immediately, the maximum reservation of the insurance can be carried out. Moreover, if it 
carries out adjustable [ of a robot's 1 rate ] according to the distance to an obstruction 14, useless transit 
time can be shortened. In the case of the latter, if the distance to an obstruction 14 is near, a robot's 1 rate 
will be lowered, and if the distance to an obstruction 14 is far, a rate will be gathered in the possible 
range 

[0033] Thus, after degenerating action on the level which can secure insurance, a robot 1 transmits the 
static-image information on the obstruction 14 obtained with CCD camera 2, and the distance 
information to the obstruction 14 obtained by the distance robot 4 to an operating set 8 by 
communication device 3a as information which shows a current situation, and becomes the remote 
operation from an operating set 8, i.e., a directions waiting state. In addition, the direction (posture) 
information of the robot 1 not only with static-image information or distance information but the 
gyroscope 7 mentioned above, the rotation information on the wheel 5 by the encoder 6, etc. can be 
transmitted to an operating set 8. Furthermore, the sound around [ which was measured with the sound- 
collecting microphone etc. ] a robot 1 may be transmitted to an operating set 8. 
[0034] By communication device 3b, an operating set 8 receives the static-image information and 
distance information which were transmitted by the robot 1, and displays a static image on the image 
display machine 9 first. Next, static-image information is analyzed on the basis of distance information, 
the 2-dimensional map information corresponding to the location shown with a static image is called 
from data storage 10, and the 2-dimensional map (line drawing) corresponding to the image display 
machine 9 with a static image is piled up and displayed. 

[0035] However, since a robot 1 does not have the fixed base and a wheel 5 may slip, even if it is 
performing dead REKONINGU processing mentioned above, the location cannot always be measured 
correctly. Therefore, if a static image and a 2-dimensional map are piled up and displayed on the image 
display machine 9, the focus or the line which carry out phase correspondence with a static image and a 
2-dimensional map may not lap correctly. In this case, suitable coordinate transformation processings, 
such as zooming, and advancing side by side and a rotation, are performed to the 2-dimensional map 
information on original, and it is made for the focus and the line of a static image and a 2-dimensional 
map which carry out phase correspondence to pile up correctly on the screen of the image display 
machine 9 with the data-conversion processor 11. 

[0036] Drawing 4 is the example which showed the static image displayed on the image display 
machine 9, and the 2-dimensional map. In this case, on the screen, a robot's 1 current location R, the 
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planned autonomous root 41, and its obstruction 42 which exists previously are shown. Moreover, in 
this example, since the edge AB of the road surface on a static image and marginal A'B' of the road 
surface to which it corresponds on a 2-dimensional map did not pile up correctly, advancing side by side 
and .rotational coordinate processing are performed to the 2-dimensional map. In addition, phase 
correspondence of the pole 43 on a screen can be carried out, and coordinate transformation processing 
can also be performed. 

[0037] Next, an operator operates a mouse 12, looking at the screen of the image display machine 9, and 
inputs the moving point which should be directed to a robot 1 . In this case, the location (point) of the 
safe autonomous orbit (henceforth, amendment orbit) which can avoid an obstruction 42 shall be 
directed, and with one point directions on a screen, when useless, two or more points are directed. For 
example, in drawing 4 , it means that the amendment orbit for avoiding an obstruction 42 was inputted 
by clicking a mouse 12 in order of Points a and b, and finally clicking the returning point C on the 
autonomous root. When shifting a robot 1 from the autonomous root 41, a robot 1 enables it to return to 
the original autonomous root 41 like this example, as the returning point C on the autonomous root 41 is 
given. You may make it search for the orbit which can return to the autonomous root 41 most smoothly 
by the operation at this time. 

[0038] Next, in an operating set 8, the coordinate data on the 2-dimensional map of each point is 
changed into the data of an amendment orbit, and the instruction data to a robot 1 with the data- 
conversion processor 1 1 in fact, and this is transmitted to a robot 1 by communication device 3b. 
[0039] By the way, in an operating set 8, in parallel to the processing for inputting the amendment orbit 
mentioned above, a suitable beep sound is emitted from a loudspeaker 13, and cautions are demanded 
from an operator. At this time, a beep sound is changed from a robot 1 according to the size of the 
distance to an obstruction 14 based on the distance information sent by the robot 1. For example, when 
distance is near, sound volume is enlarged, and sound volume is made small when far. Moreover, when 
distance is near, it is a high sound, and when far, it can be made to change [ sound / low ] (it changes 
one octave at the maximum and the minimum value of for example, detection distance). 
[0040] This beep sound is emitted until the instruction data of an amendment orbit are sent to a robot 1, 
or until there is a communication link which shows that the obstruction 14 was avoided from the robot 1 
and it returned to the original autonomous migration. 

[0041] If communication device 3 a receives the instruction data of an amendment orbit, the robot 1 
which was in the state waiting for directions from an operating set 8 will move so that an obstruction 14 
may be avoided according to the instruction data (remote migration). However, a robot 1 moves 
autonomously according to the instruction data of an amendment orbit in fact. And after being able to 
return from an amendment orbit to an original autonomous orbit, in accordance with the autonomous 
orbit of a body, autonomous migration is continued as it is. in addition, after degenerating action 
similarly when an obstruction 14 is generated ahead while performing autonomous migration according 
to the amendment orbit, it is desirable to require directions of the further amendment orbit of an operator 
— in this way In autonomous remote fusion control of this operation gestalt, autonomous migration of 
the robot 1 is carried out according to the knowledge base fundamentally memorized by the knowledge 
base section 23. When an obstruction 14 etc. is generated ahead of a robot 1 and it becomes impossible 
to be unable to respond for the information only on the knowledge base section 23, a robot's 1 action is 
degenerated and an operating set 8 is asked for subsequent directions by communication device 3a. And 
a robot 1 is moved based on the instruction data sent from the operating set 8 according to it, and after 
returning to an original autonomous orbit, it is made to perform autonomous migration succeedingly. 
Thus, since autonomous control and remote control can be united appropriately and a robot 1 can be 
moved, an operator's burden decreases compared with the case where all are depended on remote 
control, and it can respond also to generating of an obstruction 14 appropriately. Moreover, since a robot 
1 may not have the advanced autonomy of a like when performing autonomous control altogether, he 
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can build the whole including a sensing means cheaply. 

[0042] Moreover, since a robot 1 degenerates autonomous migration first and he is demanding 
directions of an operating set 8 when an obstruction 14 is generated ahead, also when people cross a 
robot's front, for example, he can fully secure the insurance of a robot 1 and each man. Since the robot 1 
is moved autonomously after that, it becomes unnecessary and to secure the communication link of real 
time between a robot 1 and a communication device 8 like [ in the case of depending all on remote 
control ] that an operating set 8 should just return directions of a robot's 1 action module unit (action 
level) like the coordinate data on a 2-dimensional map to the demand from a robot 1. Therefore, it can 
be stabilized even if communication link delay occurs between a robot 1 and an operating set 8, and a 
robot 1 can be operated. Moreover, communicative real time nature is able to use what is not necessarily 
guaranteed like PHS as communication devices 3a and 3b. 

[0043] On the other hand, since the 2-dimensional map memorized by the static image sent by the robot 
1 in the operating set 8 and data storage 10 is piled up and displayed on the image display machine 9 
when operating by remote control, an operator can grasp a robot's 1 present condition exactly. Since the 
focus path on the street, such as a manhole, the border of a road, etc. can be doubled with the static 
image and 2-dimensional map to which the 2-dimensional map was sent by the robot 1 zooming, 
advancing side by side, and by making it rotate and displaying with the data-conversion processor 1 1 at 
this time, a robot's 1 condition can be grasped easily. 

[0044] Moreover, since an operator can show a robot's 1 amendment orbit only by choosing the point 
considered as the request on the screen of the image display machine 9 using a mouse 12, he can 
perform easily remote control for making a robot 1 avoid from an obstruction 14. 
[0045] Furthermore, in the operating set 8, when there is a directions demand from a robot 1, the beep 
sound is generated from the loudspeaker 13. And since the sound volume and the frequency of a beep 
sound are changed from the robot 1 according to the distance to an obstruction 14 based on the distance 
information sent by the robot 1, an operator can judge a robot's 1 condition correctly. 
[0046] Next, a formula is shown and explained about the example of dead REKONINGU in a robot 1. 
First, when performing dead REKONINGU based on the rotation of the wheel 5 detected by the encoder 
6 (encoder method), the robot body 1000 is divided into right and left, and it is considered that it called 
it the right wheel 1101 and the left wheel 1 102, the right encoder 501, and the left encoder 502 as shown 
in drawing 8 . Moreover, it is LC about the distance between wheels of the right wheel 1 101 and the left 
wheel 1 102. It carries out. 

[0047] Amount of advance deltaER calculated from the count of the right encoder 501 in the sampling 
time at this time Amount of advance deltaEL similarly calculated from the count of the left encoder 502 
under sampling It is based and inclination deltatheta of the right and left produced during the sampling is 
shown like a degree type. 
deltatheta= (deltaER-deltaEL) / LC ~ (1) 

Moreover, the amount ER of right wheel advance And the amount EL of left wheel advance It is shown 
like a degree type. 
[0048] 
[Equation 1] 
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On the other hand, it is the initial value theta 0 of an inclination. It is based and current inclination theta 

is shown like a degree type. 

[0049] 

theta=integral deltathetadt+theta 0 - (3) 

Furthermore, amount of advance deltar to the travelling direction within the sampling time is shown like 
a degree type. 

deltar= (deltaEL+deltaER)/2 -- (4) 

In this case, it is [0050] when a formula (4) is divided into x components and y component. 
A uation 2] 

x == A r - cos & 
A y — A r * sin 0 



(5) 



A y : * >7 »J y^ft^OiifrS 

In **, the initial value of the center-of-gravity location of a body can be calculated for the present 
center-of-gravity location of a body (xG and yG) like a degree type as (xO and yO). 
[0051] 
[Equation 3] 

x c = /Axdt+x 0 j (e) 

y G =/Aydt+y 0 } 

[0052] On the other hand, when performing dead REKONINGU using a gyroscope 7 (gyroscope 
method), it is detection value thetaj from a gyroscope 7. thetaj and initial value are shown in the angular 
velocity from a gyroscope 7 like a degree type as thetajO (it is usually 0 in a start-up condition). 
[0053] 

thetaj -thetaj dt+theta jO - (7) 

It is based on this formula (7) and is a robot's 1 transit direction value xj. And the infestation direction 
value yj It asks like a degree type and things are made. 
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[0054] 
[Equation 4] 

X j = / Ar «cos dt+x 
y. = / A r • sin 0, dt+] 



50 



(8> 
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[0055] Next, a formula is shown and explained about the example when displaying a static image and a 
2-dimensional map (line drawing) on the image display machine 9 of an operating set 8. First, as shown 
in drawing 5 (a), the ground system of coordinates (X, Y, Z) of Zero O and a robot's 1 location are Zero 
OV. As indicated in drawing 5 (b) as the becoming robot system of coordinates (XV, YV, and ZV), the 
camera system of coordinates (XC, YC, ZC) by CCD camera 2 are considered. However, at robot 
system of coordinates, it is XV about a robot's 1 front. It is YV about the forward direction of a shaft, 
and the left. It is ZV about the forward direction of a shaft, and the direction of a vertical. It is made to 
correspond in the forward direction (for it to be a ground top ZV =0) of a shaft, and is XC about the 
direction of an optical axis of CCD camera 2 at camera system of coordinates. It is made to correspond 
in the forward direction of a shaft. 

[0056] Moreover, the point P on a robot map [ in / by the following explanation / ground system of 
coordinates ] (X, Y, Z) The point of robot 1 core in ground system of coordinates PG (XG, YG, and 
ZG), A robot's 1 bearing in ground system of coordinates psi (it is forward about a counterclockwise 
rotation to the X-axis), The coordinate when seeing Point P by robot system of coordinates PV (XV, 
YV, and ZV), The home position of the camera system of coordinates seen from robot system of 
coordinates PB (XB, YB, and ZB), The coordinate when seeing Point P by camera system of 
coordinates PC (XC, YC, and ZC), Coordinate PC in the flat surface (henceforth, pixel flat surface) at 
which the picture element of CCD camera 2 was constituted together with length and width The number 
of pixels in every direction within (deltax, delta y), and a pixel flat surface is expressed [ a projecting 
point ] for the scale of (x, y), and a pixel flat surface as (ixallrwidth and iyALL: length). 
[0057] First, it changes into the coordinate PV (XV, YV, and ZV) which looked at the point P on the 
robot map in ground system of coordinates (X, Y, Z) by robot system of coordinates using the degree 
type. In addition, in order to simplify count, a robot 1 is on a flat surface and assumes that it is Z= 0. 
;0058] 
Equation 5] 



Y J L 



cos 4> sin 0 
sin 0 cos 0 
0 0 
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(9) 



fcrtTU z y . z, z c =0 



Next, it changes into the coordinate PC (XC, YC, and ZC) which looked at Coordinate PV (XV, YV, 
and ZV) with the camera coordinate using the degree type. 
[0059] 
[Equation 6] 
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In this case, when a formula (9) is calculated, it comes to be shown in a degree type, and it is [0060]. 

[Equation 7] 
x t 



*Y J 



(X-X c ) cos 4> (Y-Y G ) sin 4 0 
(X-X c ) sin 0 (Y— ) cos 0 0 
0 0 0 



(11) 



[0061] 








Equation 8] 
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[0062] Therefore, based on a formula (11) and a formula (12), it is convertible for the coordinate PC 
(XC, YC, and ZC) which looked at the point P on a robot map (X, Y, Z) by camera system of 
coordinates. Next, as shown in drawing 6 , the coordinate PC (XC, YC, and ZC) searched for from the 
formula (12) is projected on a pixel flat surface, as it is shown by the degree type. 
[0063] 
[Equation 9] 
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In this case, if two-point assignment (ixl, iyl) ((ix2, iy2)) of the coordinate within the pixel flat surface 
of a pixel unit (ix and iy) is carried out, the straight line which connects two points can be made on a 
pixel flat surface. A line drawing can be displayed on the image display machine 9 as mentioned above. 
[0064] Next, the processing which puts a line drawing on a static image on the screen of the image 
display machine 9 is considered. First, actuation for putting line drawing A'B' on the part AB to which it 
corresponds on a static image is performed by the operator on a screen as shown in drawingJZ - 
[0065] That is, an operator operates a mouse 12 and clicks Point A (Point A blinks at this time). If the 
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location of Point A is right, an operator will push a carriage turn key and will decide Point A. Hereafter, 
a total of four points are decided in order of point A\ Point B, and point B'. In addition, if an error is in 
assignment of a point by actuation the middle, assignment actuation of the point will be initialized with 
an escape key. 

[0066] Thus, after deciding four points, an operator pushes a carriage-return key, makes line drawing 
A'B' advance side by side, makes it rotate further, and doubles with Part AB. That is, point A ? is made in 
agreement with Point A, and point B' is made in agreement with Point B. As a part AB, the descriptions, 
such as an angle of a road and an angle of a building, consider as an intelligible location. 
[0067] Thus, with the data-conversion processor 1 1, if an operator decides all the points A and B, A', 
and B', in order to change the point of the arbitration of a static image into the point on a 2-dimensional 
map, it will ask ifor the parameter of coordinate transformation in a procedure which is described below. 
[0068] First, it asks for whether the points A and B clicked with the mouse are equivalent to which pixel 
coordinate A [xA and yA] and B [xB and yB] on a pixel flat surface (distance unit), and asks for 
coordinate A[ on the line drawing of point A 1 and B' ] ' [XA and YA], and B' [XB and YB] further. And 
it asks for the parameter of coordinate transformation by counting relational expression backward, as 
shown below. 
[0069] 

[Equation 10] 
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Hereafter, it is referred to as theta= 0 as easiest conditions. Moreover, cosphi is written as cphi and 
sinphi is written as sphi. In this case, relational expression as shown in a degree type from a formula 
(15) - a formula (17) is obtained. 
[0070] 
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From this formula (21) [to 0071 [ furthermore, ]] 
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It becomes. However, alpha- (XA-XB) 2+(YA-YB) 2 - (25) 

It comes out. Next, it is [0072] from relational expression called cphi2+sphi2 =1. 



[Equation 13] 

(x A - x R ) 2 + (Y A - y ) 2 « 2 
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It becomes. Next, it is [0073] by substituting a formula (27) for a formula (26). 
[Equation 14] 
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It becomes. Next, it is [0074] from a formula (24), a formula (27), and the relational expression 
tanphi=sphi/cphi . 
[Equation 15] 
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It becomes. Next, it is [0075] from a formula (21). 
[Equation 161 
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(33) 



(34) 



The result to say is obtained. Since it asked for a parameter [XG and YG] required for coordinate 
transformation, and phi and f above, the focus [X, Y] of the arbitration on a line drawing is convertible 
for the point on a static image [x, y] by using each formula shown below. 
[0076] 

[Equation 17] 
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[0077] Moreover, the point [XI and Yl] of the both ends of the line on a line drawing, and [X2 and Y2] 
are changed into the point on a static image [xl and yl], and [x2 and y2] by the formula (36) - the 
formula (38). The line drawing corresponding to a static-image top can be piled up by connecting these 
points [xl and yl], and [x2 and y2]. 

[0078] Furthermore, by counting a formula (21) and a formula (27) backward, as shown in a degree 
type, coordinate transformation of the destination on a static image [x\ y'] can be carried out to the point 
on a line drawing [X 1 , Y']. 
[0079] 

[Equation 18] 
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[0080] Next, although other operation gestalten of this invention are explained, below, the same sign is 
given to the part which carries out phase correspondence with drawing 1 , and it explains focusing on 
difference with the 1st operation gestalt. 

(2nd operation gestalt) Drawing 9 is the block diagram showing the configuration of the autonomous 
remote fusion control system concerning the 2nd operation gestalt of this invention. This drawing shows 
a control-system system more to a detail. 

[0081] There are a mechanical component 100, the migration Planning Department 102, the robot 
center-of-gravity movement Planning Department 103, the movement control section 104, the servo 
control section 105, the servo driver 106, sensor I/O 107, the sensor signal transduction section 108, the 
dead REKONINGU body detection section 109, the performance information processing section 110, 
the migration information processing section 1 1 1, the exception-processing section 112, the servo 
exception-handling section 114, the kinematic-control exception-handling section 1 15, the operating 
state exception-handling section 116, the migration condition exception-handling section 117, and a 
database 121,124 in a robot side. On the other hand, there are the image display machine 9, a mouse 12, 
the intellectual remote-control section 130, the display process section 113, the movement actuation 
information processing section 118, the actuation information processing section 119 of operation, and a 
database 120 in an operating set side. Here, a robot and operating set side shall perform an exchange of 
the data through the boundary shown by the dotted line with the communication devices 3a and 3b 
shown by drawing 1 . 

[0082] In this autonomous remote fusion control, Operator MAN performs target assignment on the 2- 
dimensional map which operated the mouse 12 and was displayed on the image display machine 9. At 
this time, the 2-dimensional map beforehand recorded on the database 120 is displayed independently. 
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[0083] The intellectual remote-control section 130 carries out coordinate transformation of the target 
specified by Operator MAN to the destination on a robot map, and outputs the data A to the migration 
Planning Department 102. Here, Data A shall show a robot's 1 moving trucking by some destinations a 
and b and c— . 

[0084] The migration Planning Department 102 outputs one at a time to the robot center-of-gravity 
movement Planning Department 103 as data B with migration of the destination data a, b 5 and c 
contained in reception and Data A in the data A in which the moving trucking inputted from the 
intellectual remote-control section 130 is shown, and the robot 1 which mentions ~ later. 
[0085] The robot center-of-gravity movement Planning Department 103 plans the optimal center-of- 
gravity rate based on the data B inputted from the migration Planning Department 102, and outputs to 
the movement control section 104 as center-of-gravity rate command data C. 

[0086] The movement control section 104 is the feedback data NC inputted from the center-of-gravity 
rate command data C inputted from the robot center-of-gravity movement Planning Department 103, 
and the kinematic-control exception-handling section 15. It is based, the joint angle (an include angle, 
angular velocity) of the wheel and steering for realizing suitable center-of-gravity movement and the 
command angle of a camera mount (pan tilt) are searched for, and those data D are outputted to the 
servo control section 105. In addition, feedback data NC When those with a body which are mentioned 
later are shown, it is determined that the command angle of a wheel and a steering joint angle will 
degenerate a robot's 1 action. 

[0087] The servo control section 105 is the feedback data NS inputted from the data D inputted from the 
movement control section 104, and the- servo exception-handling section 1 14. It is based and the 
angular-velocity data E of the motor to the wheel and steering, and pan tilt which are made the optimal 
are outputted to the servo driver 106. 

[0088] The servo driver 106 receives the angular- velocity data E of the motor of a wheel and a steering, 
and a pan tilt inputted from the servo control section 105, and supplies the electrical potential difference 
or current value for generating the angular velocity which each motor of a mechanical component 100 
was ordered. As this result, a robot 1 moves and whenever [ angle-of-coverage / of a camera ] changes. 
[0089] At this time, sensor I/O 107 changes the raw sensor information F (analog data) into reception 
from a mechanical component 100, changes it into the digital data inside a calculating machine, and 
outputs it to the sensor signal transduction section 108 as sensor information G. 
[0090] The sensor signal transduction section 108 changes the sensor information G (digital data) 
inputted from sensor I/O 107 into physical quantity, such as a wheel and a steering joint angle, a 
distance sensor detection value, CCD camera information (still picture), and a pan tilt angle. And data 
MS in which a wheel, a steering joint angle, and a pan tilt angle are shown It outputs to the servo 
exception-handling section 1 14, and the data H in which a wheel and a steering joint angle, a distance 
sensor detection value, and CCD camera information (still picture) are shown are outputted to the dead 
REKONINGU object identification section 109 and the movement actuation information processing 
section 118. 

[0091] The servo exception-handling section 1 14 is the data MS in which the wheel and steering joint 
angle inputted from the sensor signal transduction section 108, and a pan tilt angle are shown. It is the 
feedback data NS as it is. It carries out and outputs to the servo control section 105. Moreover, data MS 
inputted into the database 124 based on the wheel, the abnormality data in a steering joint angle, and the 
abnormality data in a pan tilt angle which are memorized beforehand When the wheel, steering joint 
angle, and pan tilt angle which are shown are judged to be abnormalities ( for example, movable range 
over),' the abnormality blackboard 125 of an internal memory is made to memorize by making them into 
abnormality information. 

[0092] Based on the data H in which the wheel and steering joint angle inputted from the sensor signal 
transduction section 108, a distance sensor detection value, and CCD camera information (still picture) 
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are shown, the dead REKONINGU object identification section 109 asks for a robot's 1 center-of- 
gravity location from a wheel and a steering joint angle, and judges the existence of the body within 
fixed distance from a distance sensor detection value. And data MP in which the existence of a center- 
of-gravity location and a body is shown It outputs to the kinematic-control exception-handling section 
115. Moreover, the data I in which a center-of-gravity location, CCD camera information, and distance 
information are shown are outputted to the performance information processing section 110. 
[0093] The kinematic-control exception-handling section 115 is the data MP in which the existence of 
the center-of-gravity location and body which were inputted from the dead REKONINGU object 
identification section is shown. It considers as the feedback data NC as it is, and outputs to the 
movement control section 4. Moreover, data MP Data MP when the center-of-gravity location to depend 
is judged to be unusual by the database 121 as compared with the center-of-gravity malposition data 
memorized beforehand When those with a body are shown, it swerves and the abnormality blackboard 
123 which is an internal memory is made to memorize by making ** into abnormality information. 
[0094] Here, while abnormalities are detected neither in the server exception-handling section 114 nor 
the kinematic-control exception-handling section 1 15, the same processing will be repeated according to 
the moving trucking shown by the destination data a, b, and c stored in the migration Planning 
Department 102, and autonomous migration will be continued only by the robot 1 . 
[0095] By the way, in this example, the performance information processing section 110 and the 
migration information processing section 111 shall have only the function which raises through the data 
inputted from the lower level to an upper level. Moreover, the operating state exception-handling section 
116 and the migration condition exception-handling section 117 are not processing especially in this 
example. When abnormality decision of operating state and abnormality decision of a migration 
condition are required for these operating state exception-handling section 116 and the migration 
condition exception-handling section 1 1 7 ? it is for performing suitable processing in response to the data 
from the performance information processing section 110 and the migration information processing 
section 111, and a suitable function is added if needed. For example, the object identification which 
exists in the distant place of several 1 0m beyond by preparing a laser range finder as a sensor is made 
possible. When it seems that a robot 1 is moved at high speed, the laser distance information sent to the 
performance information processing section 1 1 0 from the lower level is given to the operating state 
exception-handling section 116. Laser distance information is analyzed in the operating state exception- 
handling section 1 16, if it is below a threshold, an alarm and laser distance information are given to the 
robot center-of-gravity movement control section 103 as those with a body, and it is possible to 
decelerate a center-of-gravity rate command at a predetermined rate by the robot movement control 
section 103. Moreover, it shall have only the function which raises through the data into which the 
movement actuation information processing section 118 and the actuation information processing 
section 1 19 of operation were also inputted from the lower level here to an upper level. 
[0096] Therefore, in this example, the data H outputted from the dead REKONINGU object 
identification section 119 are inputted into the exception-processing section 1 12 as data K through the 
performance information processing section 110 and the migration information processing section 111 
as it is. 

[0097] The exception-processing section 112 outputs the center-of-gravity location, CCD camera 
information, and distance information which are shown by the inputted data K to the display-processing 
section 1 13 as a robot's 1 abnormality situation data Q, when it is judged as those with a body ahead of a 
robot 1 in the kinematic-control exception-handling section 115 when a robot 1 cannot continue 
autonomous working for example. 

[0098] If the abnormality situation data Q are inputted from the exception-processing section 112, the 
display process section 113 will display the center-of-gravity location, CCD camera information, and 
distance information which are shown by the abnormality situation data Q on the image display machine 
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9, and will demand a robot's 1 remote operation from an operator. At this time, the display process 
section 113 piles up and displays the still picture shown using the CCD camera information in the 
abnormality situation data Q, and the 2-dimensional map which is inputted from the intellectual remote- 
control section 130 and by which coordinate transformation (enlarging or contracting, advancing side by 
side, rotation, etc.) was carried out. 

[0099] Then, if an amendment orbit is inputted by the operator like the 1st operation gestalt, the 
autonomous migration by the robot 1 will be resumed like the case where it mentions above based on 
the amendment orbit. 

[0100] (3rd operation gestalt) Drawing 10 is drawing showing the appearance of the autonomous remote 
fusion control system concerning the 3rd operation gestalt of this invention. This operation gestalt is an 
example of application to the outdoor monitor robot of various plants, or a works and a warehouse, (a) 
expresses an outdoor monitor robot and (b) expresses the monitor room. 

[0101] As shown in drawing 10 (a), the outdoor monitor robot 200 has a steering driving shaft and a 
transit driving shaft in each of four wheels 203, and is easily taken as that it can run also by******** 
by spot revolution, slanting migration, etc. Moreover, the path of a wheel 203 is enlarged, a suspension 
is optimized, a curbstone is overcome, and a building etc. can be approached now. Furthermore, the 
body serves as a waterproofing specification. 

[0102] The sensor 201 which has a CCD camera for telling an operator the ultrasonic sensor for 
obstruction cognition and the situation at that time is formed in the center section of a body. Moreover, 
the sensor 205 for detecting the abnormal occurrence of perimeters, such as a fire 206, is installed by the 
hand of the manipulator 204 formed in anterior part. As a sensor 205, an other unit type radiation 
thermometer, an inflammable gas sensor, etc. are used. 

[0103] Moreover, the communication device for communicating with an operating set is formed in the 
interior of a body, and the antenna 202 is installed in the body right lateral section. On the other hand, as 
shown in drawing 10 (b), the communication device 208, the loudspeaker 209, and the image display 
machine 210 are formed in the monitor room side like the 1st operation gestalt, and an operator 207 
grasps the outdoor monitor robot's 200 condition with a loudspeaker 209 and the image display vessel 
210. In addition, operating sets, such as a mouse, are omitted on the drawing. 

[0104] As shown in drawing 1 1 , this outdoor monitor robot 200 runs autonomously according to the 
round root 301 in the site set up beforehand. When the outdoor monitor robot 200 has a curbstone at this 
time, it moves by learning along with that curbstone. Moreover, in the place which does not have a 
curbstone, dead REKONINGU based on the angle of rotation of a wheel 203 is performed like the 1st 
operation gestalt, and autonomous migration is carried out, presuming a its present location. In addition, 
it learns on the basis of the pole 303 formed in the site, and it may move or you may make it presume a 
your present location using GPS Satellite 302. 

[0105] This outdoor monitor robot 200 supervises a perimeter, carrying out autonomous migration of 
301 for the round root in a site. Specifically, it is a monitor to the outbreak of gas, a fire, etc., a doubtful 
person, etc. This is performed using a sensor 201 and a sensor 205. At this time, a sensor 205 can be 
made to be able to approach an object with a manipulator 204 if needed, and that object can also be 
supervised more in a detail. In this case, even if an operator 207 operates a manipulator 204 by remote 
control, when the outdoor monitor robot 200 arrives at the location set up beforehand, you may make it 
operate a manipulator 204 autonomously. 

[0106] By the way, if a front obstruction is recognized during autonomous migration, the outdoor 
monitor robot 200 will degenerate action like the 1 st operation gestalt, for example, will stop a body. 
Then, the outdoor monitor robot 200 transmits the still picture information incorporated from the CCD 
camera to the operator 207 of the monitor interior of a room, and makes autonomous migration resume 
according to the directions of an operator 207 made according to it. By doing in this way, since an 
operator 207 should just operate it when an obstruction arises ahead of the outdoor monitor robot 200 
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(i.e., only when it becomes impossible for the outdoor monitor robot 200 to operate autonomously), 
actuation burden decreases and he can perform the monitor of various plants, or a works and a 
warehouse easily. 
[0107] 

[Effect of the Invention] When a robot performs autonomous migration based on the knowledge base set 
up beforehand fundamentally according to this invention as explained above, and it is judged that the 
autonomous migration is impossible, Until it is judged in response to the moving target which 
autonomous migration was degenerated, transmitted the situation data in which a current situation is 
shown to the operating set, and was taught in the operating set based on the situation data that the 
autonomous migration based on the knowledge base is possible again Since autonomous migration is 
performed based on the moving-target data based on an operator's remote operation, a robot can be, 
moved uniting autonomous control and remote control appropriately. Therefore, it can arrive at the 
destination correctly also by the robot which does not have advanced autonomy, being supported by the 
operator. 

[0108] Moreover, since autonomous migration is degenerated first arid it is asking for remote operation 

by the operating set when autonomous migration of a robot becomes impossible, even when an 

obstruction is generated, for example ahead of a robot, insurance can fully be secured. 

[0109] And since an operating set only teaches the moving target of an action module unit to a robot, 

there can be few burdens of an operator, and can end and the means of communications to which real 

time nature, such as PHS, is not necessarily guaranteed can also operate by remote control appropriately 

further. 



[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

l .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] It is the robot control system to which this robot is moved while performing data 
communication between a robot and an operating set. Said operating set A display means to display the 
data transmitted by the data set up beforehand and said robot in order to teach said robot, It has an 
instruction means for teaching the moving-target data of the action module unit which should be 
transmitted to said robot on the display screen of this display means. Said robot A knowledge base 
storage means by which the knowledge base for the autonomous migration set up beforehand was 
memorized, A decision means to judge whether the autonomous migration based on the knowledge base 
memorized by this knowledge base storage means is possible, When it is judged by this decision means 
that the autonomous migration based on said knowledge base is possible, autonomous migration is 
performed based on said knowledge base. When it is judged that the autonomous migration based on the 
knowledge base is impossible, after degenerating current autonomous migration, The robot control 
system characterized by having an autonomous migration means to perform autonomous migration 
based on the moving-target data to which the situation data in which a current situation is shown were 
taught by said operating set with said instruction means based on delivery and this situation data. 
[Claim 2] The robot control system according to claim 1 characterized by displaying the data transmitted 
by the data beforehand set to said indicating equipment in order to teach said robot, and said robot on 
two or more coincidence, respectively. 



[Translation done.] 
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